Image reproducing apparatus, image reproducing method, image capturing apparatus, and control method therefor

ABSTRACT

The present invention relates to an image reproducing apparatus, an image reproducing method, an image capturing apparatus, and a control method therefor. In the present invention, in order to reproduce an image signal of a captured image that is generated by performing image capture at an image rate higher than a preset image rate, an image signal to be reproduced is supplied to a signal processing unit that converts an image signal into the preset image rate, and the image rate of this image signal to be reproduced is controlled on the basis of reproduction control information generated using a motion detection result of the captured image to reduce the reproduction speed of the captured image in accordance with an increase in motion detected. An image signal of a slow motion reproduction image is output.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a national stage of International ApplicationNo. PCT/JP2008/051570 filed on Jan. 31, 2008 and claims priority toJapanese Patent Application No. 2007-022676 filed on Feb. 1, 2007, thedisclosures of which are incorporated herein by reference.

BACKGROUND

The present invention relates to an image reproducing apparatus, animage reproducing method, an image capturing apparatus, and a controlmethod therefor. More specifically, the present invention is intended toperform image capture at an image rate higher than a reference imagerate, which is set in advance, so that when an image signal of agenerated captured image of a captured image is reproduced at thereference image rate, the reproduction speed is controlled in accordancewith motion in the captured image.

Conventionally, in Japanese Unexamined Patent Application PublicationNo. 2005-295423, a method in which a captured image is temporarily heldin a memory and is further recorded on a recording medium so that slowmotion video can be reproduced in smooth motion.

That is, in a technique disclosed in Japanese Unexamined PatentApplication Publication No. 2005-295423, an image signal having a fieldfrequency of, for example, 240 fields per second, which is higher than anormal field frequency of video signal, namely, 50 fields per second or60 fields per second, is generated, and this image signal issequentially stored in a memory. Further, when image capture iscompleted, the image signal stored in this memory is read at the normalfield frequency of video signal and is recorded on a recording mediumsuch as a hard disk device. Further, when a user instructs reproduction,the image signal recorded on this recording medium is reproduced at thenormal field frequency of video signal. Therefore, a slow motionreproduction image with smooth motion can be obtained.

Incidentally, when the technique of Japanese Unexamined PatentApplication Publication No. 2005-295423 is used, since an image signalof a captured image is stored in a memory at a high speed and isthereafter read from the memory and recorded on a recording medium, thetime allowed for capturing an image with one-time image capture islimited by the capacity of the memory. For this reason, if the memoryhas a small capacity, it is difficult to hold an image signal of acaptured image to be reproduced in slow motion in the memory. Forexample, if the timing of starting writing of an image signal in thememory is earlier than that of a captured image scene to be reproducedin slow motion, there is no space in the memory when the captured imagescene to be reproduced in slow motion is written, and the image signalcannot be held. This causes a problem of missing a precious imagecapturing opportunity. Further, if the capacity of the memory isincreased so that an image signal of a captured image to be reproducedin slow motion can be easily held in the memory, cost is increased. Inaddition, in a method in which an image signal of a captured image isstored in a memory and is thereafter read and recorded on a recordingmedium, a next scene cannot be captured before all image capture resultsstored in the memory have been read. For this reason, in a case where adesired scene is repeated at short intervals, some of the repeatedscenes may be missed being captured.

Further, if an image signal of a high-speed captured image is recordedon a recording medium without the intervention of a memory, an imagesignal of a captured image to be reproduced in slow motion can berecorded on the recording medium even without controlling a recordingtiming of a captured image in accordance with memory capacity. However,if an image signal recorded in this manner is reproduced at a normalfield frequency of video signal, all captured images recorded arereproduced in slow motion. Thus, there is a case where the waiting timefor a desired slow motion image to be displayed is long.

SUMMARY Technical Problem

In consideration of the foregoing points, a feature of the presentinvention is to easily obtain a slow motion image of an object inmotion.

In order to solve the above problem, an image reproducing apparatus ofthe present invention is configured to include a signal reproducing unitthat reproduces an image signal of a captured image that is generated byperforming image capture at an image rate higher than a preset imagerate; a reproduction control information generating unit that generatesreproduction control information using a motion detection resultobtained by performing motion detection using the image signal outputfrom the signal reproducing unit; a signal processing unit that convertsan image rate of the image signal output from the signal reproducingunit into the preset image rate; and a control unit that controls theimage rate of the image signal output from the signal reproducing unitto the signal processing unit on the basis of the reproduction controlinformation to reduce a reproduction speed of the captured image inaccordance with an increase in motion indicated by the motion detectionresult.

Further, an image reproducing method of the present invention isconfigured to include a signal reproducing step of reproducing an imagesignal of a captured image that is generated by performing image captureat an image rate higher than a preset image rate; a reproduction controlinformation generating step of generating reproduction controlinformation using a motion detection result obtained by performingmotion detection using an image signal to be reproduced in the signalreproducing step; a signal processing step of converting an image rateof an image signal to be reproduced in the signal reproducing step intothe preset image rate; and a reproduction controlling step ofcontrolling the image rate of the image signal reproduced in the signalreproducing step unit, which is used in the signal processing step, onthe basis of the reproduction control information to reduce areproduction speed of the captured image in accordance with an increasein motion detected in the motion detection.

An image capturing apparatus of the present invention is configured toinclude an image capturing unit that generates an image signal of acaptured image that is generated by performing image capture at an imagerate higher than a preset image rate; a reproduction control informationgenerating unit that generates, using a motion detection result obtainedby performing motion detection using the image signal generated by theimage capturing unit, reproduction control information for reducing areproduction speed of the captured image in accordance with an increasein motion detected in the motion detection during reproduction of theimage signal; and an output unit that outputs the generated reproductioncontrol information in correspondence with the image signal.

A control method for an image capturing apparatus of the presentinvention is configured to include an image capturing step of generatingan image signal of a captured image that is generated by performingimage capture at an image rate higher than a preset image rate; areproduction control information generating step of generating, using amotion detection result obtained by performing motion detection usingthe image signal generated in the image capturing step, reproductioncontrol information for reducing a reproduction speed of the capturedimage in accordance with an increase in motion detected in the motiondetection during reproduction of the image signal; and an outputtingstep of outputting the reproduction control information incorrespondence with the image signal.

According to the present invention, a slow motion image of an object inmotion can be easily obtained.

Additional features and advantages of the present invention aredescribed in, and will be apparent from, the following DetailedDescription and the Figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram illustrating the structure of an imagecapturing apparatus.

FIG. 2 is a block diagram illustrating the structure of a correctionprocessing unit.

FIG. 3 is a block diagram illustrating the structure of a camera signalprocessing unit.

FIG. 4 is a block diagram illustrating the structure of an imagecombining unit and a frame memory unit.

FIG. 5 is a diagram for use in the explanation of image combination.

FIG. 6 is a block diagram illustrating the structure of acompression/expansion unit.

FIG. 7 is a block diagram illustrating the structure of an imagereproducing apparatus.

FIG. 8 is a block diagram illustrating the structure of an imagerecording apparatus.

FIG. 9 is a diagram for explaining a process when a captured image isrecorded in a standard image capture mode.

FIG. 10 is a diagram for explaining a process when the captured imagerecorded in the standard image capture mode is reproduced.

FIG. 11 is a diagram for explaining a process when a captured image isrecoded in a high-speed image capture mode.

FIG. 12 is a diagram for explaining a process when the captured imagerecorded in the high-speed image capture mode is reproduced.

FIG. 13 is a diagram for explaining the relationship between a motionvector and a weighting coefficient setting signal.

FIG. 14 is a diagram for explaining the relationship between an imagesignal and a weighting coefficient.

FIG. 15 is a diagram illustrating a weighting coefficient as an example.

FIG. 16 is a diagram for use in the explanation of motion detection ofan object.

FIG. 17 is a diagram illustrating the relationship between a decisionvalue and a recommended reproduction speed.

FIG. 18 is a timing chart when a captured image is recorded in thehigh-speed image capture mode.

FIG. 19 is a timing chart when the captured image recorded in thehigh-speed image capture mode is reproduced.

FIG. 20 is a timing chart when motion detection is performed andreproduction control is performed when the captured image recorded inthe high-speed image capture mode is reproduced.

FIG. 21 is a diagram illustrating the structure of another embodiment.

DETAILED DESCRIPTION

In the following, an embodiment of the present invention will beexplained with reference to the drawings. FIG. 1 is a block diagramillustrating the structure of an image capturing apparatus. An imagecapturing apparatus 10 is configured to be capable of switching between,as operation modes, a standard image capture mode and a high-speed imagecapture mode. The standard image capture mode is an operation modeduring normal image capture, in which an image signal of a referenceimage rate which is set in advance is generated. The high-speed imagecapture mode is an operation mode in which image capture is performed ata speed higher than that in the standard image capture mode so that aslow motion reproduction image having a high time resolution can beobtained to generate an image signal of an image rate which is higherthan the reference image rate and which is an integer multiple of thereference image rate (hereinafter referred to as a “high-speed imagerate”). Here, the term reference image rate refers to a field rate orframe rate used in the television system. When the image capturingapparatus 10 is in the standard image capture mode, the image capturingapparatus 10 generates an image signal of the reference image rate, forexample, an image signal of the interlaced scanning type of 60 fieldsper second or an image signal of the non-interlaced type with 60 framesper second. Further, when the image capturing apparatus 10 is in thehigh-speed image capture mode, the image capturing apparatus 10generates an image signal of a high-speed image rate, for example, animage signal of the interlaced scanning type of 240 fields per second oran image signal of the non-interlaced type of 240 frames per second.Note that in the following explanation, a case is illustrated as anexample where an image signal of the non-interlaced type of 60 framesper second (60 [fps]) is generated as an image signal of the referenceimage rate and an image signal of the non-interlaced type of 240 framesper second (240 [fps]), which is four times the reference image rate, isgenerated as an image signal in the high-speed image capture mode.

Further, in the structure of the image capturing apparatus illustratedin FIG. 1, not only a case where an image signal of a captured image isgenerated in the standard image capture mode and the high-speed imagecapture mode but also a case where an image signal can be recorded ontorecording media 62 or an image signal recorded on the recording media 62can be read to display a reproduction image or output an image signal ofa reproduction image is illustrated.

In the image capturing apparatus 10, a lens unit 11 is controlled by acontrol unit 81, which will be described below, to be capable ofchanging focus, aperture, and zoom to collect incident light on an imagecapture surface of an image sensor 121 of an image capturing unit 12.

The image capturing unit 12 of the image capturing apparatus 10 isconstructed using the image sensor 121, an AFE (Analog Front End) 122,an ADC (Analog-Digital Converter) 123, and a correction processing unit124. The operation of the image capturing unit 12 is controlled by thecontrol unit 81 which will be described below.

The image sensor 121 is implemented by a solid-state image capturingelement of the CMOS (Complementary Metal Oxide Semiconductor) type orthe like. The image sensor 121 performs a photoelectric conversionprocess on an optical image formed on the image capture surface, andoutputs an image capture signal which is an image capture result. Notethat the image sensor 121 is provided with a CDS (Correlated DoubleSampling) circuit, and this CDS circuit performs a correlated doublesampling process on the image capture signal to reduce noise in theimage capture signal.

Further, the image sensor 121 is controlled by the control unit 81 toswitch the operation thereof. When the operation mode is set to thestandard image capture mode, the image sensor 121 generates an imagecapture signal of a reference image rate (reference frame rate), forexample, an image capture signal of 60 [fps]. Further, when theoperation mode is set to the high-speed image capture mode, the imagesensor 121 generates an image capture signal of a high-speed image rate(high-speed frame rate) which is higher than the reference frame rateand which is an integer multiple of the reference frame rate, forexample, an image capture signal of 240 [fps]. Note that even in thecase of outputting an image capture signal whose frame rate is 240[fps], identically to the case of outputting an image capture signal atthe reference frame rate, the image sensor 121 reads image captureresults of all pixels from an effective image region on the imagecapture surface and outputs an image capture signal. Further, at leastone of line decimation and pixel decimation may be performed to reducethe number of pixels on one screen, as compared with a case where animage capture signal is output at the reference frame rate.

The AFE (Analog Front End) 122 performs an AGC (Automatic Gain Control)process on the image capture signal output from the image sensor 121 tocontrol the gain of the image capture signal. The ADC (Analog-DigitalConverter) 123 converts the image capture signal processed by the AFE122, which is analog, into a digital image signal, and supplies thedigital image signal to the correction processing unit 124.

The correction processing unit 124 performs various correction processeson the image signal supplied from the ADC 123. FIG. 2 is a block diagramillustrating the structure of the correction processing unit 124. Alevel correction unit 124 a of the correction processing unit 124performs a level adjustment process on the image signal supplied fromthe ADC 123. A pixel correction unit 124 b performs a pixel defectcorrection process on an image signal obtained after the leveladjustment process. A white balance adjustment unit 124 c executes awrite balance adjustment process on an image signal obtained after thepixel defect correction process, and supplies a processed image signalDV1 to a camera signal processing unit 20, a recording/reproducingprocessing unit 61, and a detection unit 71 in the manner illustrated inFIG. 1.

In the standard image capture mode and the high-speed image capturemode, the camera signal processing unit 20 performs a camera signalprocess on the image signal DV1 supplied from the image capturing unit12, and outputs a processed image signal DV2 to an imagecompression/expansion unit 41, an image output unit 51, and a displayprocessing unit 52. Further, as described below, in a case where acaptured image that is not subjected to a camera signal process isrecorded on the recording media 62 in the high-speed image capture mode,during the reproduction of the recorded captured image, the camerasignal processing unit 20 performs a camera signal process on an imagesignal DV3 supplied from the recording/reproducing processing unit 61,and outputs a processed image signal DV4 to the image output unit 51 andthe display processing unit 52.

Further, the camera signal processing unit 20 is connected to a framememory unit 31 that is constructed by, for example, an SDRAM(Synchronous Dynamic Random Access Memory) or the like. With the use ofthe frame memory unit 31, the camera signal processing unit 20 convertsthe image signal DV1 or the image signal DV3 into an image signal of thereference frame rate when the image signal DV1 or the image signal DV3has a frame rate different from the reference frame rate. That is, thecamera signal processing unit 20 outputs the image signals DV2 and DV4of the reference frame rate regardless of the frame rate of the imagesignal DV1 or the image signal DV3.

FIG. 3 is a block diagram illustrating the structure of the camerasignal processing unit 20. An image combining unit 21 is designed forthe purpose of performing image rate conversion so that the image signalDV1 or the image signal DV3 is converted into an image signal of thereference frame rate. When the operation mode is set to the standardimage capture mode, the image signal DV1 has the reference frame rate.Thus, the image combining unit 21 supplies the image signal DV1 suppliedfrom the correction processing unit 124 of the image capturing unit 12to a pixel interpolation unit 22 without converting the frame rate ofthe image signal DV1. Further, when the operation mode is set to thehigh-speed image capture mode, the image combining unit 21 performsimage rate conversion on the image signal DV1 supplied from thecorrection processing unit 124 of the image capturing unit 12 and theimage signal DV3 supplied from the recording/reproducing processing unit61 during the reproduction of the captured image recorded on therecording media 62 in the high-speed image capture mode to convert thesupplied image signals into image signals of the reference frame rate,and supplies resulting image signals to the pixel interpolation unit 22.For example, the image combining unit 21 performs image combinationusing the frame memory unit 31 to combine frame images corresponding toa reference frame period of the image signal DV1 or DV3 to produce animage of one frame. Thus, the image signal DV1 or DV3 is converted intoan image signal of the reference frame rate.

FIG. 4 is a block diagram illustrating the detailed structure of theimage combining unit 21 and the frame memory unit 31, and illustrates acase where image combination is performed using the cyclic addition ofimage signals with the use of the frame memory unit 31 to convert animage signal of the high-speed frame rate into an image signal of thereference frame rate.

The image combining unit 21 supplies the image signal DV1 supplied fromthe correction processing unit 124 or the image signal DV3 supplied fromthe recording/reproducing processing unit 61 to a multiplication circuit211. The multiplication circuit 211 multiplies the image signal DV1 orthe image signal DV3 by a weighting coefficient generated by a weightingcoefficient generation unit 212, and supplies the image signalmultiplied by the weighting coefficient to an addition circuit 213. Theaddition circuit 213 adds the image signal supplied from themultiplication circuit 211 to an image signal read from a bank of theframe memory unit 31, and supplies an image signal obtained after theadding process to the frame memory unit 31.

The frame memory unit 31 is constructed using a bank 312 a, a bank 312b, and signal switching units 311, 313, and 314. Note that the bank 312a and the bank 312 b have a memory capacity capable of storing an imagesignal of one frame.

The signal switching unit 311 supplies the image signal supplied fromthe addition circuit 213 to the bank 312 a or the bank 312 b. Note thatan image signal to be supplied to the bank 312 a is denoted by an imagesignal Dfwa and an image signal to be supplied to the bank 312 b isdenoted by an image signal Dfwb.

The signal switching unit 313 supplies an image signal Dfra read fromthe bank 312 a or an image signal Dfrb read from the bank 312 b to theaddition circuit 213. The signal switching unit 314 supplies the imagesignal Dfra read from the bank 312 a or the image signal Dfrb read fromthe bank 312 b to a division circuit 214.

Here, in the image combining unit 21 and the frame memory unit 31, thesignal switching unit 313 selects an image signal read from one bank,for example, the image signal Dfra read from the bank 312 a, and thesignal switching unit 311 supplies an image signal obtained as a resultof addition between the selected image signal Dfra and the image signalsupplied from the multiplication circuit 211 to the bank 312 a as theimage signal Dfwa. This would allow the bank 312 a to store an imagesignal subjected to cyclic addition. Further, in the image combiningunit 21 and the frame memory unit 31, when the cyclic addition of apredetermined number of frames is completed, cyclic addition isperformed using the bank 312 b, and an image signal for which the cyclicaddition of the predetermined number of frames has been completed isread from the bank 312 a and is supplied to the division circuit 214through the signal switching unit 314. The cyclic addition performed inthis manner would allow the image combining unit 21 and the frame memoryunit 31 to, while performing cyclic addition for a reference frameperiod using one bank, read an image signal for which the cyclicaddition of the predetermined number of frames has been completed fromthe other bank for the reference frame period. The read image signal canbe supplied to the division circuit 214 as an image signal of thereference frame rate.

The division circuit 214 performs a division process on the image signalsupplied from the signal switching unit 314 of the frame memory unit 31in accordance with the number of frames that have been added to theweighting coefficient. Then, an image signal for which cyclic additionhas been completed is supplied to the pixel interpolation unit 22 as animage signal within a predetermined signal level range. For example,when a case where the image signal DV1 has a constant signal level isconsidered, it is assumed that cyclic addition is performed for fourframes, each frame having a weighting coefficient of “2”. Then, thesignal level of an image signal for which cyclic addition has beencompleted is eight times the signal level of the image signal DV1. Forthis reason, the division circuit 214 performs a division processaccording to the number of frames that have been added to the weightingcoefficient, and multiplies the image signal for which cyclic additionhas been completed by (⅛) to obtain an image signal within thepredetermined signal level range.

The weighting coefficient generation unit 212 is connected to a framecounter 215. The frame counter 215 generates a count value indicatingwhich frame among the frames to be subjected to cyclic addition theimage signal to be supplied to the multiplication circuit 211corresponds to, and supplies the count value to the weightingcoefficient generation unit 212. In the case of performing cyclicaddition on the image signal DV1 supplied from the correction processingunit 124, the weighting coefficient generation unit 212 generatesweighting coefficients, which are equal for the individual frames to besubjected to cyclic addition, on the basis of a weighting coefficientselection value WS supplied from the control unit 81, and supplies theweighting coefficients to the multiplication circuit 211. Further, inthe case of performing cyclic addition on the image signal DV3 suppliedfrom the recording/reproducing processing unit 61, the weightingcoefficient generation unit 212 generates weighting coefficients, whichare equal for the individual frames to be subjected to cyclic addition,or weighting coefficients, which are different between frames, on thebasis of a weighting coefficient selection value WS supplied from thecontrol unit 81, and supplies the weighting coefficients to themultiplication circuit 211.

FIG. 5 is a diagram for explaining image combination in a case where theframe rate of the image signal DV1 or DV3 is four times the referenceframe rate. When image combination is performed using the imagecombining unit 21 and the frame memory unit 31 in the manner describedabove, as illustrated in part (A) of FIG. 5, images based on the imagesignal DV1 supplied from the correction processing unit 124 and theimage signal DV3 supplied from the recording/reproducing processing unit61 are such that, for example, four frames, namely, frames a to frame d,are combined. As illustrated in part (B) of FIG. 5, one frame (a+b+c+d)of an image of the reference frame rate is produced. Thus, the imagesignal DV1 or DV3 can be converted into an image signal of the referenceframe rate.

The pixel interpolation unit 22 in FIG. 3 converts the image signaloutput from the image combining unit 21 into image signals of threeplanes composed of R, G, and B using a demosaic process, and outputs theimage signals. A contour correction unit 23 separates high-frequencycomponents from the supplied image signals, and generates contourcorrection data from the high-frequency components. A color correctionunit 24 separates low-frequency components from the supplied imagesignals, and performs a linear matrix process or the like on thelow-frequency components to perform color correction.

A gamma/knee processing unit 25 combines the contour correction signalgenerated by the contour correction unit 23 with the image signalprocessed by the color correction unit 24, and executes gammacorrection, knee processing, or the like on the combined image signal. Acolor space conversion unit 26 converts the image signal processed bythe gamma/knee processing unit 25 into image signals of a luminancesignal and a color-difference signal. A resolution conversion unit 27converts the supplied image signals into an image signal having a presetdisplay resolution, and outputs the resulting image signal as the imagesignal DV2 or the image signal DV4.

The image compression/expansion unit 41 in FIG. 1 is configured toperform a process of performing motion detection using the suppliedimage signal, a process of compressing an image signal on the basis of amotion detection result to generate an encoded signal, and a process ofexpanding an encoded signal to generate an image signal.

In the standard image capture mode, upon receiving an instruction fromthe control unit 81 to record a captured image, the imagecompression/expansion unit 41 performs a compression process on theimage signal DV2 output from the camera signal processing unit 20 usinga moving image encoding scheme such as MPEG (Moving Picture ExpertsGroup), and outputs an encoded signal DW to the recording/reproducingprocessing unit 61. In the high-speed image capture mode, upon receivingan instruction from the control unit 81 to record a captured image, theimage compression/expansion unit 41 detects a motion vector MV from theimage signal DV2 on a macroblock-by-macroblock basis, and outputs thismotion detection result, namely, the motion vector MV, to therecording/reproducing processing unit 61 or the control unit 81.

Further, during the reproduction of a captured image recorded on therecording media 62 in the standard image capture mode, the imagecompression/expansion unit 41 performs an expansion process on anencoded signal DR supplied from the recording/reproducing processingunit 61, and supplies an obtained image signal DV5 to the image outputunit 51 or the display processing unit 52.

In addition, when the image signal DV3 is supplied from therecording/reproducing processing unit 61, the imagecompression/expansion unit 41 performs motion detection using the imagesignal DV3 to detect a motion vector MV from the image signal DV3 on amacroblock-by-macroblock basis, and outputs this motion detectionresult, namely, the motion vector MV, to the control unit 81.

FIG. 6 is a block diagram illustrating the structure of the imagecompression/expansion unit 41. In the image compression/expansion unit41, an MV detector 411 sequentially detects a motion vector MV for eachmacroblock from the image signal DV2. During recording in the standardimage capture mode, the MV detector 411 outputs this motion vector MV toa motion compensator 412. Further, during recording in the high-speedimage capture mode, the MV detector 411 outputs this motion vector MV tothe recording/reproducing processing unit 61 or the control unit 81.

During the encoding of a moving image, the motion compensator 412performs motion compensation on an image signal stored in an imagememory 413 using the motion vector MV detected by the MV detector 411,and generates and outputs a predicted value for encoding. Further,during the decoding of a moving image, similarly, the motion compensator412 generates a predicted value for decoding using a motion vector MVdecoded by an MV decoder 414.

During the encoding of a moving image, a predictive subtractor 415subtracts the predicted value for encoding, which is generated by themotion compensator 412, from the image signal DV2 input from the camerasignal processing unit 20, and outputs a prediction error value.

A DCT 416 performs a two-dimensional discrete cosine transform on theoutput signal of the predictive subtractor 415, and outputs coefficientdata obtained as a result of this process. A quantizer 417 performs aquantization process on the coefficient data generated by the DCT 416,and a variable length encoder 418 performs a variable length encodingprocess on output data of the quantizer 417. An MV encoder 419 performsan encoding process on the motion vector MV for encoding, which isdetected by the MV detector 411. A multiplexer 420 performs amultiplexing process on output data of the variable length encoder 418and output data of the MV encoder 419, and outputs a result as anencoded signal DW.

During encoding, a dequantizer 421 performs a dequantization process onoutput data of the quantizer 417 and outputs resulting data. Further,during decoding, the dequantizer 421 performs a dequantization processon output data of a variable length decoder 422, and outputs resultingdata.

An inverse DCT 423 processes the output data of the dequantizer 421 in amanner opposite to that of the DCT 416, and outputs resulting data. Anadder 424 adds the predicted value for encoding or decoding to an outputsignal of the inverse DCT 423 to generate an image signal DV5. The imagememory 413 stores and holds this decoded image signal DV5 for motioncompensation.

During decoding, a demultiplexer 426 separates the encoded signal DRoutput from the recording/reproducing processing unit 61 into a portioncorresponding to the coefficient data and a portion corresponding to themotion vector MV, and outputs the portions. The MV decoder 414 decodesthe motion vector MV from this output data of the demultiplexer 426, andoutputs the motion vector MV to the motion compensator 412. The variablelength decoder 422 decodes the coefficient data from this output data ofthe demultiplexer 426, and outputs the coefficient data to thedequantizer 421.

The image output unit 51 in FIG. 1 is an interface to which an externaldevice is connected. In the standard image capture mode and thehigh-speed image capture mode, the image output unit 51 outputs theimage signal DV2 supplied from the camera signal processing unit 20 tothe external device as an output image signal DVout. Note that theexternal device is, for example, a monitor device or the like. Further,during the reproduction of a captured image recorded on the recordingmedia 62 in the standard image capture mode, the image output unit 51outputs the image signal DV5 supplied from the imagecompression/expansion unit 41 to the external device as an output imagesignal DVout. In addition, during the reproduction of a captured imagerecorded on the recording media 62 in the high-speed image capture mode,the image output unit 51 outputs the image signal DV4 output from thecamera signal processing unit 20 to the external device as an outputimage signal DVout.

The display processing unit 52 is connected to a display unit 53 whichis constructed by, for example, an LCD (Liquid Crystal Display). In thestandard image capture mode and the high-speed image capture mode, thedisplay processing unit 52 converts the image signal DV2 output from thecamera signal processing unit 20 into a display image signal DHaccording to the display resolution of the display unit 53, and drivesthe display unit 53 using this display image signal DH so that acaptured image is displayed on the display unit 53. Further, during thereproduction of a captured image recorded on the recording media 62 inthe standard image capture mode, the display processing unit 52 convertsthe image signal DV5 output from the image compression/expansion unit 41into a display image signal DH according to the display resolution ofthe display unit 53, and drives the display unit 53 using this displayimage signal DH so that a reproduction image is displayed on the displayunit 53. In addition, during the reproduction of a captured imagerecorded on the recording media 62 in the high-speed image capture mode,the display processing unit 52 converts the image signal DV4 output fromthe camera signal processing unit 20 into a display image signal DHaccording to the display resolution of the display unit 53, and drivesthe display unit 53 using this display image signal DH so that areproduction image or a slow motion reproduction image is displayed onthe display unit 53.

The recording/reproducing processing unit 61 is controlled by thecontrol unit 81 to perform writing or reading of a captured image to orfrom the recording media 62. The recording media 62 are variousrecording media which are randomly accessible or various recording mediawhich are difficult to randomly access, such as a magnetic tape.Further, recording media which are randomly accessible are, for example,a hard disk device, a semiconductor memory such as a flash memory, anoptical disk such as a DVD (Digital Versatile Disk), and the like.

In the standard image capture mode, the recording/reproducing processingunit 61 records the encoded signal DW output from the imagecompression/expansion unit 41 onto the recording media 62. Further, inthe high-speed image capture mode, the recording/reproducing processingunit 61 records the image signal DV1 output from the image capturingunit 12 onto the recording media 62. At this time, therecording/reproducing processing unit 61 obtains, as reproductioncontrol information RJ, the motion vector MV notified from the imagecompression/expansion unit 41, or obtains reproduction controlinformation RJ generated by the control unit 81. Therecording/reproducing processing unit 61 records this reproductioncontrol information RJ in association with the image signal DV1. Thereproduction control information RJ is information for performingreproduction control for reducing the reproduction speed of a capturedimage in accordance with an increase in motion of an object in acaptured image or performing reproduction control for mitigating motionblur in a reproduction image.

During the reproduction of a captured image recorded in the standardimage capture mode, the recording/reproducing processing unit 61supplies the encoded signal DR read from the recording media 62 to theimage compression/expansion unit 41. Further, during the reproduction ofa captured image recoded in the high-speed image capture mode, therecording/reproducing processing unit 61 reads an image signal andreproduction control information RJ corresponding to the image signalfrom the recording media 62, and outputs the image signal DV3 and thereproduction control information RJ to the camera signal processing unit20 and the control unit 81, respectively.

In addition, in a case where the reproduction control information RJcannot be read during the reproduction of the captured image recorded inthe high-speed image capture mode, the recording/reproducing processingunit 61 supplies the image signal DV3 to the camera signal processingunit 20 and the image compression/expansion unit 41.

A detection unit 71 detects, for a frame period, information KDnecessary for aperture adjustment, focus adjustment, and white balanceadjustment from the image signal DV1 output from the image capturingunit 12, and notifies the control unit 81. Note that the necessaryinformation is, for example, an accumulative added value of pixel valuesin a specific region, a peak value, or the like.

The control unit 81 is connected to a user interface unit 82. The userinterface unit 82 generates an operation signal PS according to a useroperation, and supplies the operation signal PS to the control unit 81.

The control unit 81 is a microcontroller constructed by, for example, aCPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (RandomAccess Memory), and the like. The control unit 81 executes a programrecorded on the ROM or the like to generate a control signal CT so thatthe operation of the image capturing apparatus 10 can become anoperation according to a user operation based on the operation signalPS, and supplies the control signal CT to the individual portions tocontrol the operation of the individual portions. Note that in thisembodiment, the program is provided by installing it in the imagecapturing apparatus 10 beforehand; however, instead of being installedbeforehand, the program may be provided by recording it on a recordingmedium such as an optical disk, a magnetic disk, or a memory card, ormay be provided by downloading it via a network such as the Internet.

When the operation mode is set to the standard image capture mode, thecontrol unit 81 performs control so as to generate an image signal DV1of the reference frame rate and display the captured image at thereference frame rate. When the operation mode is set to the high-speedimage capture mode, the control unit 81 performs control so as togenerate an image signal DV1 of the high-speed frame rate, performconversion of the image rate, and display the captured image at thereference frame rate.

Further, in a case where it is determined that a recording startoperation has been performed when the operation mode is the standardimage capture mode, the control unit 81 further performs control toperform a compression process on an image signal of the reference framerate that has been subjected to a camera signal process and record aresulting image signal onto the recording media 62. In a case where itis determined that a recording start operation has been performed whenthe operation mode is the high-speed image capture mode, the controlunit 81 further performs control to generate reproduction controlinformation RJ for performing reproduction control for reducing thereproduction speed of the captured image in accordance with an increasein motion of an object or reproduction control for mitigating motionblur in a reproduction image, and record the reproduction controlinformation RJ in correspondence with the image signal DV1 onto therecording media 62. Note that in a case where it is determined that arecording termination operation has been performed during a recordingoperation, the control unit 81 performs control to stop recording ontothe recording media 62.

In a case where it is determined that the operation of reproducing acaptured image recorded on the recording media 62 in the standard imagecapture mode has been performed, the control unit 81 performs control toread a desired encoded signal from the recording media 62, perform adecoding process, and display a reproduction image at the referenceframe rate, or performs control to output a reproduction image signal ofthe reference frame rate.

In a case where it is determined that the operation of reproducing acaptured image recorded on the recording media 62 in the high-speedimage capture mode has been performed, the control unit 81 performscontrol to read a desired image signal and reproduction controlinformation RJ, which is in association with the image signal, from therecording media 62 and display a reproduction image at the referenceframe rate, or performs control to output an image signal of thereproduction image at the reference frame rate. Further, the controlunit 81 performs, on the basis of the reproduction control informationRJ, reproduction control for reducing the reproduction speed of thecaptured image in accordance with an increase in motion of an object orreproduction control for mitigating motion blur in the reproductionimage.

Further, in a case where the reproduction control information RJ cannotbe obtained during the reproduction of the captured image recorded onthe recording media 62 in the high-speed image capture mode, the controlunit 81 generates reproduction control information RJ from an imagesignal read from the recording media 62, and performs, on the basis ofthis reproduction control information RJ, reproduction control forreducing the reproduction speed of the captured image in accordance withan increase in motion of an object or reproduction control formitigating motion blur in a reproduction image.

Note that the control unit 81 also performs, in accordance with theinformation KD from the detection unit 71, control for performingaperture adjustment and focus adjustment of the lens unit 11 for a frameperiod and causing the correction processing unit 124 to perform whitebalance adjustment for a frame period.

Incidentally, in the embodiment described above, an image capturingapparatus in which the lens unit 11, the image capturing unit 12, thecamera signal processing unit 20, the image compression/expansion unit41, the recording/reproducing processing unit 61, and the like areprovided as an integrated unit has been explained. However, the lensunit 11, the image capturing unit 12, the display processing unit 52,the display unit 53, and the detection unit 71 may be provided asseparate units to construct an image reproducing apparatus or an imagerecording apparatus.

FIG. 7 is a block diagram illustrating the structure of an imagereproducing apparatus 90. Note that in FIG. 7, portions corresponding tothose of FIG. 1 are assigned the same numerals.

The image reproducing apparatus 90 includes a recording/reproducingprocessing unit 61 for reading an encoded signal, an image signal, orthe like from recording media 62, an image compression/expansion unit 41for performing an expansion process on a read encoded signal DR, acamera signal processing unit 20 that performs a signal process using animage signal DV3 read from the recording media 62, a frame memory unit31, an image output unit 51 for outputting an image signal DV4 outputfrom the camera signal processing unit 20 or an image signal DV5 outputfrom the image compression/expansion unit 41 to an external device, acontrol unit 81 that controls the operation of the individual portions,and a user interface unit 82.

In the image reproducing apparatus 90, when the reading of an encodedsignal recorded on the recording media 62 is performed, therecording/reproducing processing unit 61 supplies a read encoded signalDR to the image compression/expansion unit 41. Note that the encodedsignal is a signal obtained by performing a compression process on animage signal of the reference frame rate. The imagecompression/expansion unit 41 performs an expansion process on theencoded signal DR supplied from the recording/reproducing processingunit 61, and supplies an obtained image signal DV5 to the image outputunit 51.

Further, in the image reproducing apparatus 90, when the reading of animage signal of the high-speed frame rate which is recorded on therecording media 62 is performed, the recording/reproducing processingunit 61 reads an image signal and reproduction control information RJcorresponding to the image signal. The camera signal processing unit 20performs a process of converting the image signal DV3 supplied from therecording/reproducing processing unit 61 into an image signal of thereference frame rate or a camera signal process. The image output unit51 outputs the image signal DV4 of the reference frame rate as an outputimage signal DVout. Further, in the image reproducing apparatus 90, in acase where control of the reproduction speed or control for mitigatingmotion blur is performed on the basis of the reproduction controlinformation RJ, the control unit 81 controls the operation of the camerasignal processing unit 20 or the recording/reproducing processing unit61 on the basis of the reproduction control information RJ, and controlsthe frame rate of the image signal DV3 to be supplied to the camerasignal processing unit 20 or a weighting coefficient used in the imagecombining unit 21 of the camera signal processing unit 20.

Further, in the image reproducing apparatus 90, in a case where thereproduction control information RJ cannot be obtained when the readingof an image signal of the high-speed frame rate which is recorded on therecording media 62 is performed, the recording/reproducing processingunit 61 supplies a read image signal DV3 to the camera signal processingunit 20. The camera signal processing unit 20 performs a process ofconverting the image signal DV3 supplied from the recording/reproducingprocessing unit 61 into an image signal of the reference frame rate or acamera signal process. The image output unit 51 outputs the image signalDV4 of the reference frame rate as an output image signal DVout.Further, in the image reproducing apparatus 90, in a case wherereproduction control for reducing the reproduction speed of a capturedimage in accordance with an increase in motion of an object orreproduction control for mitigating motion blur in a reproduction imageis performed, the recording/reproducing processing unit 61 supplies theread image signal DV3 to the image compression/expansion unit 41. Theimage compression/expansion unit 41 performs motion detection using theimage signal DV3, and supplies an obtained motion vector MV to thecontrol unit 81. The control unit 81 controls the operation of therecording/reproducing processing unit 61 on the basis of the motionvector MV so that the frame rate of the image signal DV3 to be suppliedto the camera signal processing unit 20 can be changed, thus reducingthe reproduction speed of a captured image in accordance with anincrease in motion of an object. Further, the control unit 81 controls aweighting coefficient used in the image combining unit 21 of the camerasignal processing unit 20 on the basis of the motion vector MV tomitigate motion blur in a reproduction image.

FIG. 8 is a block diagram illustrating the structure of an imagerecording apparatus 91. Note that in FIG. 8, portions corresponding tothose of FIG. 1 are assigned the same numerals.

The image recording apparatus 91 includes a camera signal processingunit 20 that performs various camera signal processes on an imagesignal, a frame memory unit 31, an image compression/expansion unit 41that performs a compression process on an image signal to generate anencoded signal, a recording/reproducing processing unit 61 for writingan encoded signal DW into the recording media 62, a control unit 81 thatcontrols the operation of the individual portions, and a user interfaceunit 82.

In the image recording apparatus 91, when an image signal DV1 of thereference frame rate is recorded, the image compression/expansion unit41 performs a compression process on an image signal DV2 of thereference frame rate, which has been subjected to a camera signalprocess using the camera signal processing unit 20, to generate anencoded signal DW. The recording/reproducing processing unit 61 performsa process of writing the encoded signal DW generated by the imagecompression/expansion unit 41 into the recording media 62.

Further, when an image signal DV1 of the high-speed frame rate isrecorded, the camera signal processing unit 20 performs not only acamera signal process but also image combination using the frame memoryunit 31, and converts the image signal DV1 of the high-speed frame rateinto an image signal of the reference frame rate. The imagecompression/expansion unit 41 generates a motion vector MV using theimage signal DV2 of the reference frame rate which is output from thecamera signal processing unit 20. The recording/reproducing processingunit 61 obtains, as reproduction control information RJ, the motionvector MV generated by the image compression/expansion unit 41, orobtains reproduction control information RJ generated by the controlunit 81 on the basis of the motion vector MV. Then, therecording/reproducing processing unit 61 performs a process of writingthis reproduction control information RJ into the recording media 62 inassociation with the image signal DV1 of the high-speed frame rate.

Next, an image capturing operation, an operation of recording a capturedimage, and an operation of reproducing a recorded captured image will beexplained. FIG. 9 illustrates a process when the operation mode is setto the standard image capture mode. Note that in FIG. 9 and FIGS. 10 to12, which will be described below, in order to clarify which block inFIGS. 1 and 7 each process corresponds to, the processes are assignedthe numerals of corresponding blocks.

An image capturing apparatus performs the generation of an image capturesignal to generate an image capture signal of a reference frame ratewhich is, for example, 60 [fps]. The image capturing apparatus performsAFE/ADC processes to adjust the signal level of the image capture signaland thereafter convert the image capture signal into a digital signal.The image capturing apparatus performs a correction process to generatean image signal DV1 that has been subjected to signal level or whitebalance adjustment and pixel defect correction. The image capturingapparatus performs a camera signal process to generate an image signalDV2 that has been subjected to contour correction, color correction,gamma correction, knee correction, and the like. Further, the imagecapturing apparatus performs a display process to convert the imagesignal DV2 into a display image signal DH having a display resolutioncorresponding to that of the display unit 53. Note that the imagecapturing apparatus performs a detection process to detect informationnecessary for aperture adjustment, focus adjustment, and white balanceadjustment.

The image capturing apparatus performs the above processes to display acaptured image on a screen of the display unit 53. Therefore, a user cancheck what object has been captured by viewing the image displayed onthe screen of the display unit 53. Further, if the image capturingapparatus outputs an output image signal DVout, the captured image canbe displayed using an external device.

Next, when a captured image is recorded, the image capturing apparatus(image recording apparatus) performs an image compression process togenerate an encoded signal DW. In addition, the image capturingapparatus (image recording apparatus) performs a recording process ofrecording the encoded signal DW onto the recording media 62. That is, animage signal DV1 captured at the reference frame rate is encoded andrecorded on the recording media 62.

FIG. 10 illustrates a process when an image recorded on the recordingmedia 62 by performing the process illustrated in FIG. 9 is reproduced.The image capturing apparatus (image reproducing apparatus) performs areproduction process to read a desired encoded signal from the recordingmedia 62. The image capturing apparatus (image reproducing apparatus)performs an image expansion process to generate an image signal DV5 fromthe encoded signal. In addition, the image capturing apparatus performsa display process to convert the image signal DV5 into a display imagesignal DH having a display resolution corresponding to that of thedisplay unit 53. The image capturing apparatus performs (imagereproducing apparatus) the above processes to display a reproductionimage on the screen of the display unit 53. Further, if the imagecapturing apparatus (image reproducing apparatus) outputs an outputimage signal DVout, the reproduction image can be displayed using anexternal device.

FIG. 11 illustrates a process when the operation mode is set to thehigh-speed image capture mode.

The image capturing apparatus performs the generation of an imagecapture signal to generate an image capture signal of a high-speed framerate which is, for example, 240 [fps]. The image capturing apparatusperforms AFE/ADC processes to adjust the signal level of the imagecapture signal and thereafter convert the image capture signal into adigital signal. The image capturing apparatus performs a correctionprocess to generate an image signal DV1 that has been subjected tosignal level or white balance adjustment and pixel defect correction.The image capturing apparatus performs a camera signal process on theimage signal DV1 of the high-speed frame rate. In the camera signalprocess, not only contour correction, color correction, gammacorrection, knee correction, and the like, but also image combinationfor converting an image signal of the high-speed frame rate into animage signal of the reference frame rate is performed, and an imagesignal DV2 of the reference frame rate is generated.

Further, the image capturing apparatus performs a display process toconvert the image signal DV2 into a display image signal DH having adisplay resolution corresponding to that of the display unit 53. Theimage capturing apparatus performs the above processes to display acaptured image on the screen of the display unit 53 even when thehigh-speed image capture mode is set. Further, if the image capturingapparatus outputs an output image signal DVout, the captured image canbe displayed using an external device. Note that the image capturingapparatus is designed such that the frame rate is set higher than thereference frame rate, and aperture adjustment, focus adjustment, andwhite balance adjustment processes are performed for a frame period.Thus, the control accuracy and speed of them can be improved as comparedwith those in the standard image capture mode.

Next, when a captured image is recorded, the image capturing apparatus(image recording apparatus) performs an image compression process togenerate a motion detection result, for example, a motion vector MV. Inaddition, the motion vector MV is set to reproduction controlinformation RJ, or reproduction control information RJ is generated onthe basis of the motion vector MV. In addition, the image capturingapparatus (image recording apparatus) performs a recording process torecord an image signal of the high-speed frame rate and the reproductioncontrol information RJ onto the recording media 62 in association witheach other.

FIG. 12 illustrates a process when an image recorded on the recordingmedia 62 by performing the process illustrated in FIG. 11 is reproduced.The image capturing apparatus (image reproducing apparatus) performs areproduction process to read a desired image signal DV3 and reproductioncontrol information RJ associated with the image signal from therecording media 62. The image capturing apparatus (image reproducingapparatus) performs a reproduction control process to cause theperformance of a process of generating a slow motion image so that theframe rate of the image signal DV3 is controlled on the basis of thereproduction control information RJ to reduce the reproduction speed inaccordance with an increase in motion of an object in a captured imageand a process of generating a slow motion image in which blur of anobject that moves a large amount is mitigated using a camera signalprocess. The image capturing apparatus (image reproducing apparatus)performs a camera signal process to generate an image signal DV4 of thereference frame rate, which has been subjected to processes such ascontour correction, color correction, gamma correction, and kneecorrection, from the image signal DV3. Further, the image capturingapparatus (image reproducing apparatus) performs, when performing acamera signal process to convert the image signal DV3 into an imagesignal of the reference frame rate, image combination so that motionblur can be mitigated. In addition, the image capturing apparatusperforms a display process to convert the image signal DV4 into an imagesignal having a display resolution corresponding to that of the displayunit 53. The image capturing apparatus performs the above processes todisplay a reproduction image on the screen of the display unit 53.Further, if the image capturing apparatus (image reproducing apparatus)outputs an output image signal DVout, the reproduction image can bedisplayed using an external device.

Further, in a case where when the reproduction control information RJassociated with the desired image signal DV3 cannot be obtained,reproduction control for reducing the reproduction speed of a capturedimage in accordance with an increase in motion of an object orreproduction control for mitigating motion blur in a reproduction imageis performed, the image capturing apparatus (image reproducingapparatus) performs motion detection using the image signal DV3 in thereproduction control process, and causes the performance of a process ofgenerating a slow motion image so that the frame rate of the imagesignal DV3 is controlled on the basis of an obtained motion vector toreduce the reproduction speed in accordance with an increase in motionof an object in a captured image or a process of generating a slowmotion image in which blur of an object that moves a large amount ismitigated using a camera signal process.

Note that the control unit 81 of the image capturing apparatus (imagereproducing apparatus) may be configured to control the operation of therecording/reproducing processing unit 61 so that image signals recordedon the recording media 62 are sequentially reproduced in accordance witha reproduction speed specified by a user. For example, in a case where auser specifies the 1× reproduction speed, the control unit 81 performsthe reading of an image signal from the recording media 62 so that theframe rate of the image signal DV3 can be 240 [fps], sequentiallyperforms image combination on four consecutive frames of the imagesignal DV3 using the camera signal processing unit 20, and performsconversion into an image signal of the reference frame rate, namely, 60[fps]. Further, in a case where a user specifies the (½)× reproductionspeed, the control unit 81 performs the reading of an image signal fromthe recording media 62 so that the frame rate of the image signal DV3can be 120 [fps], sequentially performs image combines on fourconsecutive frames of the image signal DV3 using the camera signalprocessing unit 20, and performs conversion into an image signal of thereference frame rate, namely, 60 [fps]. Further, in a case where a userspecifies the (¼)× reproduction speed, the control unit 81 performs thereading of an image signal from the recording media 62 so that the framerate of the image signal DV3 can be equal to the reference frame rate,namely, 60 [fps].

Further, in a case where a user specifies fast-forward reproduction,that is, a reproduction speed more than 1×, the control unit 81 reads animage signal from the recording media 62 by performing decimation inaccordance with the reproduction speed to generate an image signal ofthe reference frame rate which indicates an image having the specifiedreproduction speed. Also in this case, images of a plurality ofconsecutive frames may be combined to generate an image of one frame infast-forward reproduction.

Next, an operation of mitigating motion blur in a reproduction imagewill be explained. In a case where image combination is performed on theimage signal DV3 to convert the image signal DV3 into an image signal ofthe reference frame rate, for example, as illustrated in FIG. 5described above, in a case where the images of the four frames a to dare combined to generate an image of one frame (a+b+c+d), a large amountof motion of an object can cause large difference in position of theobject in the frames a to d. For this reason, if cyclic addition isperformed by giving an equal weighting coefficient, an image of anobject that moves a large amount is blurred. Therefore, a high-qualityreproduction image cannot be obtained.

Accordingly, the control unit 81 determines a weighting coefficientselection value WS for generating weighting coefficients, which aredifferent between frames, for a portion with a large amount of motion sothat a plurality of frames to be subjected to cyclic addition contain aframe having a large weight and a frame which is temporally distant fromthis frame and which is given a small weight. The control unit 81supplies the weighting coefficients to the weighting coefficientgeneration unit 212 of the image combining unit 21 in the camera signalprocessing unit 20.

FIG. 13 is a diagram for explaining an operation of generating aweighting coefficient selection value to be supplied from the controlunit 81 to the weighting coefficient generation unit 212 of the imagecombining unit 21.

The control unit 81 determines a weighting coefficient selection valueWS in accordance with the magnitude of motion which is obtained byperforming motion detection using an image signal. For example, a motionvector MV obtained when the image compression/expansion unit 41 performsa compression process on the image signal DV1 is used to set a weightingcoefficient selection value WS. Specifically, the absolute value of themotion amount of the motion vector MV (hereinafter referred to as a“motion-vector absolute value”) MVm is compared with a threshold value,and a weighting coefficient selection value WS is determined inaccordance with a comparison result. The threshold value may beimplemented by a fixed value regardless of the number of frames to besubjected to image combination, or the number of threshold values may bechanged in accordance with the number of frames to be subjected to imagecombination. For example, when the number of frames to be subjected toimage combination is large, the number of threshold values is increasedso that a multiplicity of weighting coefficients can be set inaccordance with the motion-vector absolute value MVm.

Part (A) of FIG. 13 illustrates a weighting coefficient selection valueWS obtained in a case where, for example, captured images which arecaptured at 240 [fps] are combined in units of four frames and combinedimages are reproduced at 60 [fps], that is, when the reproduction speedis set to 1×. Note that in part (A) of FIG. 13, two threshold valuesMta1 and Mta2 are provided.

When the motion-vector absolute value MVm is less than the thresholdvalue Mta1, the control unit 81 sets the weighting coefficient selectionvalue WS to “Wa0”. Further, when the motion-vector absolute value MVm isequal to or greater than the threshold value Mta1 and is less than thethreshold value Mta2, the control unit 81 sets the weighting coefficientselection value WS to “Wa1”. In addition, when the motion-vectorabsolute value MVm is equal to or greater than the threshold value Mtb2,the control unit 81 sets the weighting coefficient selection value WS to“Wa2”.

Part (B) of FIG. 13 illustrates a weighting coefficient selection valueWS obtained in a case where, for example, captured images which arecaptured at 240 [fps] are combined in units of two frames and combinedimages are reproduced at 60 [fps], that is, when the reproduction speedis set to (½)×. Note that in part (B) of FIG. 13, one threshold valueMtb1 is provided.

When the motion-vector absolute value MVm is less than the thresholdvalue Mtb1, the control unit 81 sets the weighting coefficient selectionvalue WS to “Wb0”. Further, when the motion-vector absolute value MVm isequal to or greater than the threshold value Mtb1, the control unit 81sets the weighting coefficient selection value WS to “Wb1”.

Here, the motion vector MV is detected for each macroblock. Thus, asillustrated in FIG. 14, the control unit 81 sets a weighting coefficientselection value WS for each macroblock of the image signal DV3 inaccordance with the motion-vector absolute value MVm. Note that in FIG.14, a case is illustrated as an example where the amount of motion issmall at a peripheral portion of the screen and where the closer to thecenter, the larger the amount of motion gradually becomes. The weightingcoefficient selection value WS is set to “Wa0” for a macroblock in theperipheral portion of the screen, and the weighting coefficientselection value WS is set to “Wa1” for a macroblock in a portion closerto the center. For a macroblock in the center, the weighting coefficientselection value WS is set to “Wa2”.

The weighting coefficient generation unit 212 of the image combiningunit 21 sequentially outputs weighting coefficients according to theweighting coefficient selection value WS for each frame within a periodinvolved in image combination. FIG. 15 illustrates weightingcoefficients as an example. In FIG. 15, “×1” shows weightingcoefficients obtained in a case where captured images that are capturedat, for example, 240 [fps] are subjected to image combination in unitsof four frames and a combined image is reproduced at 60 [fps], that is,when the reproduction speed is set to 1×. In this case, four frames areto be subjected to image combination. Thus, weighting coefficients areset for four frames a to d to be subjected to image combination.

Further, in FIG. 15, “×½” shows weighting coefficients obtained in acase where captured images that are captured at, for example, 240 [fps]are subjected to image combination in units of two frames and a combinedimage is reproduced at 60 [fps], that is, when the reproduction speed isset to (½)×. In this case, two frames are to be subjected to imagecombination. Thus, weighting coefficients are set for two frames a and bto be subjected to image combination.

The weighting coefficient generation unit 212 sets a weightingcoefficient of “2” for the frames a, b, c, and d to be subjected toimage combination when the weighting coefficient selection value WS is“Wa0”, and sets a weighting coefficient of “2” for the frames a and b tobe subjected to image combination when the weighting coefficientselection value WS is “Wb0”. That is, images of portions with no motionor a small amount of motion are given an equal weight. Therefore, sincean image signal obtained after cyclic addition is performed is changedto an image signal within a predetermined signal level range using thedivision circuit 214 even if noise is superimposed on an image of one offrames to be subjected to image combination, an image signal output fromthe image combining unit 21 is an image signal from which noise has beenreduced in a portion with no motion or a portion with a small amount ofmotion.

When the weighting coefficient selection value WS is “Wa1”, theweighting coefficient generation unit 212 sets weighting coefficients of“1, 1, 3, 3” for the frames a, b, c, and d to be subjected to imagecombination in the frame order. When the weighting coefficient selectionvalue WS is “Wa2”, the weighting coefficient generation unit 212 setsweighting coefficients of “1, 1, 2, 4” for the frames a, b, c, and d tobe subjected to image combination in the frame order. Further, when theweighting coefficient selection value WS is “Wb1”, the weightingcoefficient generation unit 212 sets weighting coefficients of “1, 3”for the frames a and b to be subjected to image combination in the frameorder. That is, weighting coefficients are set for a portion with alarge amount of motion in such a manner that frames to be subjected toimage combination contain a frame having a large weight and a framewhich is temporally distant from this frame and which is given a smallweight. Further, weighting coefficients are set for a portion with alarger amount of motion so that the difference in weight between a framehaving a large weight and a temporally distant frame having a smallweight becomes larger. For this reason, since a frame which istemporally distant from a frame given a large weight, that is, a framein which the position of an object is largely different, is given asmall weight, an image signal with reduced motion blur can be obtainedas compared with a case where image combination is performed by givingan equal weight.

Note that only in a case where a user specifies a process of settingweighting coefficients which are different between frames for individualframes to be subjected to image combination so that motion blur can bereduced, the control unit 81 may execute the process. That is, normally,frames to be subjected to image combination may be given an equalweighting coefficient. Then, an image capture result obtained byperforming image capture at a high speed may be reproduced with an imagequality similar to that with which an image capture result obtained byperforming image capture in the standard image capture mode isreproduced.

Further, the control unit 81 may be configured to execute a process ofsetting weighting coefficients which are different between frames forindividual frames to be subjected to image combination in a portion witha large amount of motion during recording in the high-speed imagecapture mode so that the reproduction control information RJ can includea weighting coefficient selection value or weighting coefficients. Inthis case, weighting coefficients can be easily set during reproduction.For example, even an image reproducing apparatus having no function forsetting a weighting coefficient from a motion vector MV can output animage signal of the reference frame rate in which motion blur has beenreduced.

Further, the control unit 81 may be configured such that in a case wherethe reproduction control information RJ cannot be obtained during thereproduction of a captured image recorded on the recording media 62, theimage compression/expansion unit 41 performs motion detection using theimage signal DV3 and a weighting coefficient selection value WS isdetermined from a motion vector MV obtained by this motion detection.

Next, a case where the reproduction speed is controlled in accordancewith motion of an object will be explained. In response to a user'sinstruction, the control unit 81 determines a recommended reproductionspeed SR so that the reproduction speed can be reduced as the amount ofmotion of an object in a captured image increases, and displays thisrecommended reproduction speed SR on the display unit 53 in an on-screendisplay manner. Further, in a case where a user instructs reproductionat the recommended reproduction speed SR, the control unit 81 controlsthe operation of the individual portions so that an image signal DV4having the recommended reproduction speed SR can be generated. Inaddition, the control unit 81 detects a motion of a background on whichmotion due to camera works such as panning and tilting has no influence,and calculates the magnitude of relative motion of an object withrespect to the background using this motion of the background todetermine a recommended reproduction speed SR in accordance with thecalculated magnitude of relative motion of the object.

FIG. 16 is a diagram for explaining a process of determining arecommended reproduction speed. The control unit 81 sets a frame BF on acaptured image so that the motion of the background and the motion of adesired object can be distinguished from each other, and separates thecaptured image into a center portion where the desired object is locatedand a peripheral portion b corresponding to a background portion.Further, the control unit 81 produces the absolute value of a motionvector MV detected using an image signal of the captured image, andcalculates a motion-vector absolute value MVm indicating the magnitudeof motion. Next, the control unit 81 separates the motion-vectorabsolute value MVm into parts inside and outside the frame BF andperforms addition. Then, the control unit 81 subtracts an absolute valuesum obtained by adding motion-vector absolute values MVm outside theframe BF from an absolute value sum obtained by adding motion-vectorabsolute values MVm inside the frame BF as given in Equation (1) tocalculate a decision value MU indicating the magnitude of relativemotion of the desired object with respect to the background.MU=Σ(motion-vector absolute values for individual macroblocks within theframe)−Σ(motion-vector absolute values for individual macroblocksoutside the frame)  (1)

Next, the control unit 81 compares the decision value MU with thresholdvalues AVt1 and AVt2, and determines a recommended reproduction speed SRon the basis of a comparison result. FIG. 17 is a diagram illustratingthe relationship between a decision value and a recommended reproductionspeed.

When the decision value MU is less than the threshold value AVt1, thecontrol unit 81 sets the recommended reproduction speed SR to “Fr0”which is 1× (×1). Further, when the decision value MU is equal to orgreater than the threshold value AVt1 and is less than the thresholdvalue AVt2, the control unit 81 sets the recommended reproduction speedSR to “Fr1”. In addition, when the decision value MU is equal to orgreater than the threshold value AVt2, the control unit 81 the controlunit 81 sets the recommended reproduction speed SR to “Fr2”. Here, in acase where the image signal DV1 of the high-speed frame rate has 240[fps] and the reference frame rate is 60 [fps], when the recommendedreproduction speed SR is “Fr2”, the control unit 81 sets thereproduction speed for which the reading of an image signal from therecording media 62 is performed so that the frame rate of the imagesignal DV3 can be 60 [fps] and for which the image signal DV3 isprocessed without being subjected to image combination to produce animage signal of a reproduction image, that is, (¼)×, as the recommendedreproduction speed SR. Further, when the recommended reproduction speedSR is “Fr1”, for example, the control unit 81 sets an intermediate valuebetween the 1× and (¼)×, namely, (½)×, as the recommended reproductionspeed SR.

If the recommended reproduction speed SR is set in this manner, the 1×reproduction speed is obtained when the magnitude of relative motion ofthe object with respect to the background is small, and the reproductionspeed is reduced in accordance with an increase in motion. For thisreason, if reproduction is performed at the recommended reproductionspeed SR, the reproduction speed can be automatically changed in such amanner that a motion reproduction image in which the object slowly movescan be obtained when the amount of motion of the object with respect tothe background is large while the motion of the object can follow theactual motion when the amount of motion of the object is small.Therefore, it is not necessary for a user to perform an operation ofswitching to the slow motion reproduction operation when the amount ofmotion of the object is large so that the user can check the motion ofthe object using video played in slow motion and switching to 1× speedreproduction when the amount of motion of the object becomes small.Thus, user usability can be significantly improved.

Further, the control unit 81 may execute the setting of the recommendedreproduction speed SR during recording in the high-speed image capturemode so that the reproduction control information RJ can include therecommended reproduction speed SR. In this case, the recommendedreproduction speed SR can be easily set during reproduction. Forexample, even an image reproducing apparatus having no function forsetting a recommended reproduction speed SR from a motion vector MV canautomatically change the reproduction speed in accordance with motion ofan object.

Further, the control unit 81 may also execute the setting of therecommended reproduction speed SR during the reproduction of an imagerecorded in the high-speed image capture mode. In this case, duringrecording in the high-speed image capture mode, the control unit 81records the reproduction control information RJ on the recording media62 in correspondence with the image signal DV1 so that the reproductioncontrol information RJ can include the recommended reproduction speedSR. Further, during the reproduction of an image recorded in thehigh-speed image capture mode, the control unit 81 reads an image signaland the reproduction control information RJ corresponding to the imagesignal, and controls the frame rate to be supplied to the camera signalprocessing unit 20 so that the recommended reproduction speed SRindicated by this reproduction control information RJ can be obtained.

Further, the control unit 81 may be configured such that in a case wherethe reproduction control information RJ cannot be obtained during thereproduction of the captured image recorded on the recording media 62,the image compression/expansion unit 41 performs motion detection usingthe image signal DV3 and the recommended reproduction speed SR isdetermined from a motion vector MV obtained by this motion detection tocontrol the reproduction speed of the captured image.

FIG. 18 is a time chart illustrating an operation when an image signalis recorded using the image capturing apparatus 10 and the imagerecording apparatus 91. Note that in FIG. 18, a case is illustratedwhere a recommended reproduction speed SR is determined on the basis ofa motion vector MV and where reproduction control information RJincluding this recommended reproduction speed SR is recorded incorrespondence with the image signal.

Part (A) of FIG. 18 is a vertical synchronization signal VD of thereference frame rate, namely, 60 [fps]. Part (B) of FIG. 18 is avertical synchronization signal VDc used by the image capturing unit 12to generate an image signal DV1. The image capturing apparatus 10 causesthe image sensor 121, the AFE 122, the ADC 123, the correctionprocessing unit 124, and the detection unit 71 to operate insynchronization with the vertical synchronization signal VDc to generatean image signal DV1 illustrated in part (C) of FIG. 18. Note that inpart (C) of FIG. 18, consecutive numbers represent individual frames.

Part (H) of FIG. 18 is a vertical synchronization signal VDs of thereference frame rate on which the operations performed in the camerasignal processing unit 20, the image compression/expansion unit 41, theimage output unit 51, the display processing unit 52, and the displayunit 53 are based. Further, vertical synchronization signals VDh and VDsare assumed to be signals synchronized with the vertical synchronizationsignal VD.

In the standard image capture mode, the image capturing unit 12generates a signal of frame “0” of the image signal DV1 synchronizedwith the vertical synchronization signal VDc which is set at thereference frame rate. Since the image signal DV1 is set at the referenceframe rate, the camera signal processing unit 20 performs camera signalprocesses other than image combination. The signal of frame “0” of animage signal DV2 output from this camera signal processing unit 20 issupplied to the display unit 53 through the display processing unit 52.Accordingly, as illustrated in part (J) of FIG. 18, a display imagesignal DH to be supplied to the display unit 53 or an output imagesignal DVout from the image output unit 51 shows an image of frame “0”.

In the high-speed image capture mode, the image capturing unit 12generates signals of frames “1”, “2”, . . . of the image signal DV1synchronized with the vertical synchronization signal VDc of 240 [fps],which is four times 60 [fps]. Since the image signal DV1 is set at thehigh-speed frame rate, the camera signal processing unit 20 performsimage combination and performs conversion into an image signal of thereference frame rate.

Parts (D) to (G) of FIG. 18 are signals for explaining image combinationperformed by the image combining unit 21 of the camera signal processingunit 20. Part (D) of FIG. 18 illustrates an image signal Dfwa to bewritten in the bank 312 a of the frame memory unit 31, part (E) of FIG.18 illustrates an image signal Dfra to be read from the bank 312 a ofthe frame memory unit 31, part (F) of FIG. 18 illustrates an imagesignal Dfwb to be written in the bank 312 b of the frame memory unit 31,and part (G) of FIG. 18 illustrates an image signal Dfrb to be read fromthe bank 312 b of the frame memory unit 31.

The image combining unit 21 stores the signal of frame “1” in the bank312 a. For the next frame, the image combining unit 21 reads the signalof frame “1” stored in the bank 312 a and adds it to the signal of frame“2”. Then, a signal of cyclic-addition frame “2′” is stored in the bank312 a. Note that in parts (D) to (G) of FIG. 18, a cyclic-addition framethat has been subjected to image combination is represented by a numberwith the “′” symbol.

In this manner, the image combining unit 21 performs image combinationusing cyclic addition. When image combination for a reference frameperiod is completed, that is, when the signal of cyclic-addition frame“4′” obtained by performing cyclic addition on frames “1” to “4” isstored in the bank 312 a, the image combining unit 21 switches thewriting of an image signal from the bank 312 a to the bank 312 b.Further, the image combining unit 21 reads a signal obtained after imagecombination for four frames which is stored in the bank 312 a, whilereturning the signal to a signal within a predetermined level rangeusing the division circuit, in synchronization with the verticalsynchronization signal VDs, and performs a camera signal process such ascontour correction. Thereafter, a resulting signal is supplied to theimage output unit 51 or the display unit 53 through the displayprocessing unit 52.

The image combining unit 21 stores the signal of frame “5” in the bank312 b. For the next frame, the image combining unit 21 reads the signalof frame “5” stored in the bank 312 b and adds it to the signal of frame“6”. Then, the image combining unit 21 stores a signal ofcyclic-addition frame “6′” in the bank 312 b. When the signal ofcyclic-addition frame “8′” obtained by performing cyclic addition onframes “5” to “8” is stored in the bank 312 b, the image combining unit21 switches the bank in which an image signal is to be written to thebank 312 a. Then, the image combining unit 21 reads a signal obtainedafter image combination for four frames which is stored in the bank 312b, while returning the signal to a signal within a predetermined signallevel range using the division circuit, for a reference frame period,and performs a process such as contour correction. Thereafter, aresulting signal is supplied to the image output unit 51 or the displayunit 53 through the display processing unit 52. Subsequently, processingis performed in a similar manner. Then, as illustrated in part (J) ofFIG. 18, even in the high-speed image capture mode, an image can bedisplayed or output at the reference frame rate.

Further, during a recording operation, the control unit 81 records animage signal generated in the standard image capture mode, for example,the signal of frame “0”, on the recording media 62 after performing acompression process, and records an image signal generated in thehigh-speed image capture mode, for example, the signals of frames “1”,“2”, . . . , on the recording media 62 without performing a camerasignal process or a compression process.

In addition, in the high-speed image capture mode, the control unit 81causes the image compression/expansion unit 41 to process an imagesignal output from the camera signal processing unit 20 to determine therecommended reproduction speed SR illustrated in part (I) of FIG. 18 onthe basis of motion information generated by the imagecompression/expansion unit 41. Then, the control unit 81 recordsreproduction control information RJ including this recommendedreproduction speed SR on the recording media 62 in correspondence withan image signal generated in the high-speed image capture mode. Notethat in a case where a weighting coefficient is set during reproductionalthough it is not illustrated in the figure, the weighting coefficientselection value WS is included in the reproduction control informationRJ and is recorded on the recording media 62 in correspondence with theimage signal.

FIG. 19 is a time chart illustrating an operation when an image recordedon the recording media 62 is generated using the image capturingapparatus 10 and the image reproducing apparatus 90. Note that in FIG.19, a case is illustrated where a user has instructed reproduction atthe recommended reproduction speed.

Part (A) of FIG. 19 is a vertical synchronization signal VD of thereference frame rate, namely, 60 [fps]. Part (B) of FIG. 19 illustratesa recommended reproduction speed SR read together with an image signalfrom the recording media 62. Part (C) of FIG. 19 illustrates a verticalsynchronization signal VDr of an image signal to be read from therecording media 62. Further, part (I) of FIG. 19 is a verticalsynchronization signal VDs of the reference frame rate on which theoperations performed in the camera signal processing unit 20, the imagecompression/expansion unit 41, the image output unit 51, the displayprocessing unit 52, and the display unit 53 are based.

When the read recommended reproduction speed SR is “Fr0” which is 1×(×1), the control unit 81 sets the image signal DV3 at the frame rateduring recording, that is, the image signal DV3 at 240 [fps], andsupplies the image signal DV3 to the camera signal processing unit 20.Part (D) of FIG. 19 illustrates the image signal DV3.

Since the image signal DV3 has been recorded in the high-speed imagecapture mode, the control unit 81 causes the camera signal processingunit 20 to perform image combination to generate an image signal of thereference frame rate.

Parts (E) to (H) of FIG. 19 are signals for explaining image combinationperformed by the image combining unit 21 of the camera signal processingunit 20. Part (E) of FIG. 19 illustrates an image signal Dfwa to bewritten in the bank 312 a of the frame memory unit 31, part (F) of FIG.19 illustrates an image signal Dfra to be read from the bank 312 a ofthe frame memory unit 31, part (G) of FIG. 19 illustrates an imagesignal Dfwb to be written in the bank 312 b of the frame memory unit 31,and part (H) of FIG. 19 illustrates an image signal Dfrb to be read fromthe bank 312 b of the frame memory unit 31.

The image combining unit 21 performs image combination in the mannerdescribed above. When image combination for a reference frame period iscompleted, that is, when the signal of cyclic-addition frame “4′”obtained by performing image combination on frames “1” to “4” is storedin the bank 312 a, the image combining unit 21 switches the bank inwhich an image signal is to be written from the bank 312 a to the bank312 b. Further, in image combination, weighting coefficients are set inaccordance with motion information, and frames to be subjected to cyclicaddition are weighted using the set weight coefficients. Further, theimage combining unit 21 reads a signal obtained after image combinationfor four frames which is stored in the bank 312 a, while returning thesignal to a signal within a predetermined level range using the divisioncircuit, in synchronization with the vertical synchronization signalVDs, and performs a camera signal process such as contour correction.Thereafter, a resulting signal is supplied to the image output unit 51or the display unit 53 through the display processing unit 52. Part (J)of FIG. 19 illustrates a display image signal DH to be supplied to thedisplay unit 53 or an output image signal DVout from the image outputunit 51.

Next, when the read recommended reproduction speed SR is “Fr1 (×½)”which is (½)×, the control unit 81 sets the image signal DV3 at a framerate which is (½) times that during recording, that is, the image signalDV3 at 120 [fps], and supplies the image signal DV3 to the camera signalprocessing unit 20. Also at this time, since the frame rate of the imagesignal DV3 is higher than the reference frame rate, the control unit 81causes the camera signal processing unit 20 to perform image combinationto generate an image signal of the reference frame rate.

That is, the image combining unit 21 stores the signal of frame “9” inthe bank 312 a. For the next frame, the image combining unit 21 readsthe signal of frame “9” stored in the bank 312 a and adds it to thesignal of frame “10”. Then, a signal of cyclic-addition frame “10′” isstored in the bank 312 a.

When image combination for a reference frame period is completed, thatis, when the signal of cyclic-addition frame “10′” obtained byperforming image combination on frames “9” and “10” is stored in thebank 312 a, the image combining unit 21 switches the bank in which animage signal is to be written from the bank 312 a to the bank 312 b.Further, the image combining unit 21 reads a signal obtained after imagecombination for two frames which is stored in the bank 312 a, whilereturning the signal to a signal within a predetermined level rangeusing the division circuit, in synchronization with the verticalsynchronization signal VDs, and performs a process such as contourcorrection. Thereafter, a resulting signal is supplied to the imageoutput unit 51 or the display unit 53 through the display processingunit 52.

Next, when the read recommended reproduction speed SR is “Fr2 (×¼)”which is (¼)×, the control unit 81 sets the image signal DV3 at a framerate which is (¼) times that during recording, that is, the image signalDV3 at 60 [fps], and supplies the image signal DV3 to the camera signalprocessing unit 20. At this time, since the frame rate of the imagesignal DV3 is the reference frame rate, the control unit 81 supplies theimage signal DV3 to the image output unit 51 or the display unit 53through the display processing unit 52 after performing a camera signalprocess such as contour correction, without performing imagecombination, using the camera signal processing unit 20. Note that thecamera signal processing unit 20 is configured such that when imagecombination is performed, image signals for which image combination fora reference frame period has been performed are read for the nextreference frame period. Thus, a signal obtained after image combinationis delayed one frame with respect to the image signal DV3. Therefore,when the image signal DV3 has the reference frame rate, the control unit81 controls the image signal DV3 to be written in the bank 312 a or thebank 312 b and the written image signal to be read for the nextreference frame period. Even when the recommended reproduction speed SRis switched, the reproduction of an image is sequentially performedusing the image signal DV3 of each frame.

In this manner, if the reproduction operation is performed at therecommended reproduction speed SR, the image signal DV3 is subjected toimage combination in units of four frames when the amount of motion issmall to produce an image of the reference frame rate. Therefore, theimage to be displayed or output can automatically be a 1× reproductionimage. Further, in accordance with an increase in motion, the frame rateof the image signal DV3 is reduced and the number of frames to besubjected to image combination is also reduced. Therefore, the image tobe displayed or output can automatically be a slow motion reproductionimage.

Further, the control unit 81 sets an equal weighting coefficient whenthe amount of motion is small, and sets weighting coefficients which aredifferent between frames to be subjected to image combination so thatmotion blur caused by image combination can be reduced when the amountof motion is large. Therefore, a section with a small amount of motionis an image with noise reduced, and a section with a large amount ofmotion is an image with reduced blur even if image combination isperformed.

FIG. 20 illustrates a timing chart in case where motion detection isperformed using an image signal read from the recording media 62, arecommended reproduction speed is set, and a user instructs reproductionat this recommended reproduction speed.

Part (A) of FIG. 20 is a vertical synchronization signal VD of thereference frame rate, namely, 60 [fps]. Part (B) of FIG. 20 illustratesa vertical synchronization signal VDr of an image signal read from therecording media 62, and the vertical synchronization signal VDr is setat a frame period of 240 [fps] which is the frame rate of the recordedimage signal. Further, part (I) of FIG. 20 is a vertical synchronizationsignal VDs of the reference frame rate on which the operations performedin the camera signal processing unit 20, the image compression/expansionunit 41, the image output unit 51, the display processing unit 52, andthe display unit 53 are based.

Part (D) of FIG. 20 illustrates an image signal DV3 to be supplied tothe camera signal processing unit and the image compression/expansionunit 41, and part (C) of FIG. 20 illustrates a recommended reproductionspeed SR that is determined by performing motion detection using theimage signal DV3.

The recording/reproducing processing unit 61 performs the reading of animage signal from the recording media 62, and supplies, insynchronization with the vertical synchronization signal VD and thevertical synchronization signal VDr, an image signal DV3 for one frameas an image signal of a frame of interest to the camera signalprocessing unit 20 and the image compression/expansion unit 41. Theimage compression/expansion unit 41 performs motion detection using theimage signal of the frame of interest supplied in synchronization withthe vertical synchronization signals VD and VDr and an image signal of areference frame which is, for example, one frame previous to this frame,and detects a motion vector MV. The control unit 81 determines arecommended reproduction speed SR and a weighting coefficient selectionvalue WS of a reference frame period including this frame of interest onthe basis of the detected motion vector MV.

For example, the image compression/expansion unit 41 performs motiondetection using the image signal of frame “1” which is the frame ofinterest and the image signal of frame “0” which is the reference frame,and detects a motion vector MV. The control unit 81 determines arecommended reproduction speed SR of a reference frame period includingframe “1” on the basis of this motion vector MV. Here, when a decisionvalue MU calculated on the basis of the detected motion vector MV isless than the threshold value AVt1, the control unit 81 sets therecommended reproduction speed SR of the reference frame periodincluding frame “1” to “Fro” which is 1× (×1). Since image combinationof four frames of the image signal DV3 is performed for 1×, the controlunit 81 controls the recording/reproducing processing unit 61 to supplythe image signals of the three frames subsequent to frame “1” to thecamera signal processing unit 20 and the image compression/expansionunit 41.

Parts (E) to (H) of FIG. 20 are signals for explaining image combinationperformed by the image combining unit 21 of the camera signal processingunit 20. Part (E) of FIG. 20 illustrates an image signal Dfwa to bewritten in the bank 312 a of the frame memory unit 31, part (F) of FIG.20 illustrates an image signal Dfra to be read from the bank 312 a ofthe frame memory unit 31, part (G) of FIG. 20 illustrates an imagesignal Dfwb to be written in the bank 312 b of the frame memory unit 31,and part (H) of FIG. 20 illustrates an image signal Dfrb to be read fromthe bank 312 b of the frame memory unit 31.

The image combining unit 21 performs image combination in the mannerdescribed above. When image combination for a reference frame period iscompleted, that is, when the signal of cyclic-addition frame “4′”obtained by performing image combination on frame “1” to “4” is storedin the bank 312 a, the image combining unit 21 switches the bank inwhich an image signal is to be written from the bank 312 a to the bank312 b. Further, in image combination, weighting coefficients are set inaccordance with a weighting coefficient selection value WS determined onthe basis of a detected motion vector, and frames to be subjected tocyclic addition are weighted with the set weight coefficients. Further,the image combining unit 21 reads a signal obtained after imagecombination for four frames which is stored in the bank 312 a, whilereturning the signal to a signal within a predetermined level rangeusing the division circuit, in synchronization with the verticalsynchronization signal VDs for the next reference frame period, andperforms a process such as contour correction. Thereafter, a resultingsignal is supplied to the image output unit 51 or the display unit 53through the display processing unit 52. Part (J) of FIG. 20 illustratesa display image signal DH to be supplied to the display unit 53 or anoutput image signal DVout to be output from the image output unit 51.Note that when the weighting coefficient selection value WS isdetermined, the image signal of the first frame to be subjected tocyclic addition has already been stored in a bank. For this reason, ifan equal coefficient is given t,o the first frame in motion coefficientsthat are selected in accordance with the weighting coefficient selectionvalue WS, motion coefficients for the second and subsequent frames canbe adjusted and motion blur can be mitigated. For example, in FIG. 15,it is assumed that coefficients of “1, 1, 1, 1” are set when theweighting coefficient selection value WS is “Wa0”. In the divisioncircuit 214 illustrated in FIG. 4, when the weighting coefficientselection value WS is “Wa0”, the signal level of the image signalsupplied from the frame memory unit 31 is multiplied by (¼). This wouldallow the coefficient for the frame a to be set to “1” when theweighting coefficient selection value WS is either of “Wa0”, “Wa1”, and“Wa2”. Thus, even if the image signal of the first frame to be subjectedto cyclic addition has already been stored in a bank, noise in a portionwith no motion or a portion with a small amount of motion, or motionblur in a portion with a large amount of motion can be reduced.

Next, motion detection is performed using the image signal of frame “5”which is the frame of interest and the image signal of frame “4” whichis the reference frame, and the recommended reproduction speed SR of areference frame period including frame “5” is determined to be “Fro”which is 1× (×1) on the basis of the detected motion vector MV. Then,cyclic addition for four frames is performed using the bank 312 b, andthe generation of the display image signal DH or the output of theoutput image signal DVout is performed.

Next, motion detection is performed using the image signal of frame “9”which is the frame of interest and the image signal of frame “8” whichis the reference frame, and a recommended reproduction speed SR of areference frame period including frame “8” is determined on the basis ofthe detected motion vector MV. Here, when the decision value MU is equalto or greater than the threshold value AVt1 and is less than thethreshold value AVt2, the recommended reproduction speed SR of thereference frame period including frame “8” is set to “Fr1” which is (½)×“×½”. Since image combination of two frames of the image signal DV3 isperformed for (½)×, the control unit 81 controls therecording/reproducing processing unit 61 to supply the image signal forone frame subsequent to frame “8” to the camera signal processing unit20 and the image compression/expansion unit 41.

The image combining unit 21 performs image combination. When imagecombination for a reference frame period is completed, that is, when thesignal of clic-addition frame “10′” obtained by performing imagecombination on frames “9” and “10” is stored in the bank 312 a, theimage combining unit 21 switches the bank in which an image signal is tobe written from the bank 312 a to the bank 312 b. In image combination,frames to be subjected to cyclic addition are weighted with weightingcoefficients that are set in accordance with a weighting coefficientselection value WS determined on the basis of a detected motion vector.Further, the image combining unit 21 reads a signal obtained after imagecombination for two frames which is stored in the bank 312 a, whilereturning the signal to a signal within a predetermined level rangeusing the division circuit, in synchronized with the verticalsynchronization signal VDs for the next reference frame period, andperforms a process such as contour correction. Thereafter, a resultingsignal is supplied to the image output unit 51 or the display unit 53through the display processing unit 52.

Further, motion detection is performed using the image signal of frame“13” which is the frame of interest and the image signal of frame “12”which is the reference frame. When the decision value MU based on thedetected motion vector MV is equal to or greater than the thresholdvalue AVt2, the recommended reproduction speed SR of a reference frameperiod including frame “13” is set to “Fr2” which is (¼)× “×¼”. Sinceimage combination for the image signal DV3 is not performed for (¼)×,the image signal of frame “13” which is stored in the bank 312 a is readin synchronization with the vertical synchronization signal VDs for thenext reference frame period, and is subjected to a camera signal processsuch as contour correction. Thereafter, a resulting signal is suppliedto the image output unit 51 or the display unit 53 through the displayprocessing unit 52.

In this manner, motion detection is performed using an image signal readfrom the recording media 62, and the recommended reproduction speed SRor the weighting coefficient selection value WS is determined on thebasis of a motion detection result. Thus, even if reproduction controlinformation RJ has not been recorded on the recording media 62, imagecombination can be performed on the image signal DV3 in units of fourframes when the amount of motion is small, and an image of the referenceframe rate is obtained. Therefore, the image to be displayed or outputcan automatically be a 1× reproduction image. Further, in accordancewith an increase in motion, the frame rate of the image signal DV3 isreduced and the number of frames to be subjected to image combination isalso reduced. Therefore, the image to be displayed or output canautomatically be a slow motion reproduction image.

Further, when the amount of motion is small, image combination isperformed by giving an equal weighting coefficient. When the amount ofmotion is large, weighting coefficients which are different betweenframes to be subjected to image combination are set so that motion blurcan be reduced in an image obtained after combination. Therefore, asection with a small amount of motion in a reproduction image is animage with noise reduced, and a section with a large amount of motion isan image with reduced blur even if image combination is performed.

According to the embodiment described above, therefore, a recommendedreproduction speed is determined in accordance with motion of an objectin a captured image. Thus, if reproduction is performed at therecommended reproduction speed, when the amount of motion of an objectis large, the reproduction speed is automatically reduced and a slowmotion reproduction image is displayed without causing a user to searchfor a position where a captured image with a large amount of motion ofan object is recorded and perform slow motion reproduction. Userconvenience can be improved.

Further, if a recording operation in the high-speed image capture modeis started beforehand so that a slow motion reproduction image having ahigh time resolution can be obtained without missing an image capturingopportunity, in a case where a slow motion reproduction image isobtained by reducing the reproduction speed of a captured image recordedin the high-speed image capture mode, the waiting time for an image witha desired motion to be displayed in slow motion becomes significantlylong. However, according to this invention, when the amount of motion issmall, reproduction is performed at, for example, 1× speed. When theamount of motion is large, the reproduction speed is automaticallyreduced, resulting in a slow motion reproduction image. This can preventthe waiting time for an image with a desired motion to be displayed inslow motion from being long.

Further, image signals for a reference frame period are weighted inaccordance with motion of an image and are subjected to an additionprocess so that an image signal of the high-speed frame rate isconverted into an image signal of the reference frame rate. Thus, ahigh-quality slow motion image in which the SN ratio of a still portionor a portion with a small amount of motion is improved and in which bluris prevented from occurring in a portion with a large amount of motioncan be obtained.

Further, a motion vector is determined using motion detection, and aweight or a recommended reproduction speed is determined in accordancewith the motion amount of this motion vector. Thus, a weight or arecommended reproduction speed can easily and reliably be determined.

Further, in the image capturing apparatus described above, in thestandard image capture mode, the image signal DV1 is subjected to acompression process and stored in recording media. Thus, the amount ofsignal of a captured image to be recorded can be saved. Further,reproduction control information RJ is generated from a motion vectorobtained by the image compression/expansion unit 41 that performs acompression process on the image signal DV1. Thus, it is not necessaryto separately provide a motion detection unit for generating thereproduction control information RJ. Thus, simplification inconfiguration of the image capturing apparatus as well as cost reductioncan be achieved.

Note that the image capturing apparatus described above is not limitedto that for performing recording and reproduction of a captured image.For example, an image signal of a high-speed frame rate and reproductioncontrol information RJ may be output from the image output unit 51 inassociation with each other.

Further, in the image capturing apparatus described above, a case hasbeen illustrated where a camera signal process or a compression processis performed using an image signal of a reference frame rate. However,in a case where a camera signal processing unit or an imagecompression/expansion unit has high processing capabilities so that acamera signal process or a compression process can be performed using animage signal of a high-speed frame rate, an image signal of a high-speedframe rate that has been subjected to a camera signal process, or thissignal may be subjected to a compression process, may be recorded on therecording media 62 or output from the image output unit 51.

Further, in the embodiment described above, a motion vector generatedwhen the image compression/expansion unit 41 performs a compressionprocess is used as a result of motion detection of an object. However,motion detection can be performed using a wide variety of methods suchas, for example, a method of determining the magnitude of variance withrespect to the average amount of motion on a screen.

FIG. 21 is a block diagram illustrating another embodiment of thepresent invention. An image capturing apparatus 95 is provided with twosystems of camera signal processing units 20 a and 20 b for monitoringand external output. The image capturing apparatus 95 is constructed inthe same manner as that of the image capturing apparatus 10, except thatthe configuration regarding the two systems of camera signal processingunits 20 a and 20 b is different.

The image capturing apparatus 95 is connected to a display device 97via, for example, a home network 96. In the image capturing apparatus95, a control unit 81 performs data communication at the time ofconnection through an image output unit 51 to detect an allowablemaximum frame rate (frame frequency) of the input of the display device97. Note that the detection of the maximum frame rate is executed byobtaining reproduction control information regarding the display device97 from the display device 97 in a data communication process betweenthe image capturing apparatus 95 and the display device 97 which isexecuted at the time of connection.

The control unit 81 switches the frame rate of image signals DV2 b andDV4 b output from the camera signal processing unit 20 b for externaloutput so that an image signal of this obtained maximum frame rate canbe output from the image output unit 51.

That is, in a case where the maximum frame rate in the display device 97is 60 [fps] (the maximum frame frequency is 60 [Hz]), the operation ofthe camera signal processing unit 20 b is controlled in the same manneras that of the camera signal processing unit 20 to output image signalsDV2 b and DV4 b of 60 [fps].

In a case where the maximum frame rate in the display device 97 is 120[fps], for an image signal DV3 reproduced from the recording media 62 at240 [fps] and an image signal DV1 obtained in the high-speed imagecapture mode, cyclic addition of two consecutive frames is performed andgenerated image signals DV2 b and DV4 b of 120 [fps] are output to thedisplay device 97.

Further, for an image signal DV3 reproduced from the recording media 62at 120 [fps], the frame rate is not reduced, and the image signal DV3 isprocessed using the camera signal processing unit 20 b and is output tothe display device 97. Further, for an image signal DV3 reproduced fromthe recording media 62 at 60 [fps] or an image signal DV1 output fromthe image capturing unit 12 in the standard image capture mode, an imagesignal of an identical frame is repeatedly output alternately using thebanks 312 a and 312 b to up-convert the image signal DV3 to 120 [fps],and the resulting image signal is output to the display device 97.

Note that the up-conversion process can be implemented by, instead ofperforming a process of simply repeating an identical frame, using awide variety of techniques such as the case of performing frameinterpolation based on motion compensation using a motion vector.Further, instead of up-conversion, the operation of the image capturingunit 12 may be switched to generate an image signal DV1 of 120 [fps].

The camera signal processing unit 20 a on the monitoring side receivesan image signal DV3 reproduced from the recording media 62, and outputsthe image signal DV3 at 60 [fps] which is the frame rate supported bythe display processing unit 52 and the display unit 53.

Note that, instead of providing two systems of camera signal processingunits themselves, two systems of only image combining units 21 may beprovided. In this case, however, a gamma process and the like are firstexecuted and, finally, an image combination process is executed.

In this manner, in a case where a captured image recorded in thehigh-speed image capture mode is reproduced from a recording medium at aframe rate higher than at least a frame rate that can be supported by anexternal device, depending on this external device, the frame rate isreduced to a frame rate that can be supported by the external device andthe captured image is output. Therefore, an image capture result can beoutput with higher image quality.

Incidentally, in a case where an image signal is transmitted via anetwork such as a home network, the case of transmission of a signalfrom another device via this network is also predictable. Thus, in acase where the transmission speed of the network is low, there is a casewhere the image signal cannot be transmitted in real time. Accordingly,in a case where an image signal read from the recording media 62 isoutput to the display device 97, the frame rate of the image signal tobe output to the display device 97 is dynamically switched so as to becompatible with the recommended reproduction speed to prevent the imagesignal DV4 b from being output to the display device 97 at anunnecessarily high frame rate. Note that this embodiment is configuredin the same manner as the image capturing apparatus 10 described above,except that the frame rate of the image signal to be output to thedisplay device 97 is different.

In the high-speed image capture mode, the control unit 81 determines arecommended reproduction speed using a motion vector detected by theimage compression/expansion unit 41. Further, in a case where thisdetermined recommended reproduction speed is 1×, four consecutive framesof an image signal DV1 whose frame rate is 240 [fps] are sequentiallysubjected to image combination using the camera signal processing unit20 b to generate an image signal DV2 b whose frame rate is 60 [fps].This image signal DV2 b is output to the display device 97.

In a case where the recommended reproduction speed is (½)×, twoconsecutive frames of an image signal DV1 whose frame rate is 240 [fps]are sequentially subjected to image combination using the camera signalprocessing unit 20 b to generate an image signal DV2 b whose frame rateis 120 [fps]. This image signal DV4 b is output to the display device97. Further, in a case where the recommended reproduction speed is (¼)×,the frame rate of an image signal DV1 whose frame rate is 240 [fps] isnot reduced by the camera signal processing unit 20 b, and an imagesignal DV2 b of 240 [fps] is output to the display device 97.

During the reproduction of a captured image recorded on the recordingmedia 62 in the high-speed image capture mode, in a case where a userinstructs reproduction at 1× speed, the reading from the recording media62 is performed so that the frame rate of the image signal DV3 can be240 [fps]. Further, the camera signal processing unit 20 b converts theframe rate of this reproduced image signal DV3 in accordance with arecommended reproduction speed. Therefore, also in this case, in caseswhere recommended reproduction speeds are 1×, (½)×, and (¼)×, the camerasignal processing unit 20 b generates image signals DV4 b whose framerates are 60 [fps], 120 [fps], and 240 [fps], respectively, and outputsthe image signals DV4 b to the display device 97.

During the reproduction of an image signal recorded on the recordingmedia 62 in the high-speed image capture mode, in a case where a userinstructs reproduction at (½)× speed, the reading from the recordingmedia 62 is performed so that the frame rate of the image signal DV3 canbe 120 [fps]. Further, in a case where the recommended reproductionspeed is 1×, the frame rate of the image signal DV3 is converted into 60[fps] using the camera signal processing unit 20 b, and the resultingimage signal DV3 is output to the display device 97. In contrast, incases where recommended reproduction speeds are (½)× and (¼)×, an imagesignal DV4 b whose frame rate is 120 [fps] is output to the displaydevice 97 without the frame rate being reduced.

During the reproduction of an image signal recorded on the recordingmedia 62 in the high-speed image capture mode, in a case where a userinstructs reproduction at (¼)× speed, the reading from the recordingmedia 62 is performed so that the frame rate of the image signal DV3 canbe 60 [fps]. Further, an image signal DV4 b of 60 [fps] is output to thedisplay device 97 with the frame rate being reduced.

During the reproduction of an image signal recorded on the recordingmedia 62 in a high-time-resolution slow motion image capture mode, in acase where a user instructs reproduction at a recommended reproductionspeed, the image combination process performed by the camera signalprocessing unit 20 b is interrupted, and an image signal DV4 b of aframe rate which is reproduced from the recording media 62 is output tothe display device 97.

Note that in the standard image capture mode, also during thereproduction of a captured image recorded on the recording media 2 inthe standard image capture mode, similarly, a recommended reproductionspeed may be detected and the frame rate may be dynamically changed inaccordance with this recommended reproduction speed. Further, thedynamic switching between frame rates in this standard image capturemode may be executed by up-converting an image signal of 60 [fps] or maybe executed by switching the operation frequency of the image sensor 121itself.

In this manner, the frame rate of an image signal to be output to anexternal device is dynamically switched in accordance with motion of anobject. This can sufficiently reduce the amount of signal of an imagesignal to be sent to a network while ensuring practically sufficientimage quality. Therefore, even in a case where the network is sharedwith another device, an image signal can be transmitted in real time.

Further, the frame rate of an image signal to be output to an externaldevice may not necessarily be dynamically changed in accordance with arecommended reproduction speed. Alternatively, the dynamic switching maybe performed in accordance with a network which is a transmission pathof the image signal and/or in accordance with an external device so asto ensure real-time transmission of the image signal.

Note that the switching between frame rates in accordance with a networkcan be executed by, for example, when sending an image signal to thenetwork on a packet-by-packet basis, determining the so-called degree ofcongestion in the network based on the waiting time for anacknowledgement to be obtained after the network is requested to send apacket, and reducing the frame rate more for higher congestion.

Further, the switching between frame rates in accordance with anexternal device can be executed by, for example, when sending an imagesignal to the external device on a packet-by-packet basis, counting thewaiting time for an acknowledgement of transmission to be obtained afterthe external device is requested to transmit an image signal, and moregreatly reducing the frame rate for a longer waiting time.

Note that in the embodiment described above, a case has been describedwhere the frame rate of an image signal is reduced using cyclicaddition. However, the present invention is not limited thereto, and theframe rate can be reduced by simply performing an addition process oncorresponding image signals of a plurality of frames. In addition, in acase where a practically sufficient image quality can be ensured, theframe rate can be reduced using frame decimation.

Further, in the embodiment described above, a case has been describedwhere the reference frame rate is set to 60 [fps] and where, in thehigh-speed image capture mode, an image signal DV1 having a frame rateof 240 [fps] which is four times the reference frame rate is generated.However, the present invention is not limited thereto, and can be widelyapplied to cases where image capture is performed at various framerates, such as, for example, a case where an image signal DV1 having aframe rate which is three times the reference frame rate is generated.

Further, in the embodiment described above, a case has been describedwhere an image obtained by performing image capture in the high-speedimage capture mode is reproduced at 1× speed, (½)× speed, and (¼)×speed: However, the present invention is not limited thereto, and canalso be widely applied to a case where, for example, the image isreproduced at (⅓)× speed and other cases.

In addition, in the embodiment described above, a case has beendescribed where an image signal of the non-interlaced type is generated.However, the present invention is not limited thereto, and can also bewidely applied to a case where an image signal of the interlaced type isgenerated.

Industrial Applicability

The present invention is suitable for, for example, performing imagecapture at an image rate higher than a normal image rate of video signalto generate an image signal and obtaining a slow motion reproductionimage using this image signal.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope of the present invention andwithout diminishing its intended advantages. It is therefore intendedthat such changes and modifications be covered by the appended claims.

The invention claimed is:
 1. An image reproducing apparatus comprising: a signal reproducing unit that reproduces an image signal of a captured image that is generated by performing image capture at an image rate higher than a preset image rate; a reproduction control information generating unit that generates reproduction control information using a motion detection result obtained by performing motion detection using the image signal output from the signal reproducing unit; a signal processing unit that converts an image rate of the image signal output from the signal reproducing unit into the preset image rate by combining frames of the image signal; and a control unit that controls the image rate of the image signal supplied from the signal reproducing unit to the signal processing unit on the basis of the reproduction control information to reduce a reproduction speed of the captured image in accordance with an increase in motion indicated by the motion detection result, wherein the control unit sets weighting coefficients for respective frames of the image signal based on the motion detection result, and the signal processing unit combines the frames according to the weighting coefficients, and wherein the respective frames of the image signal are weighted with at least one of at least two weighting coefficients.
 2. The image reproducing apparatus according to claim 1, wherein the reproduction control information generating unit determines a recommended reproduction speed from a motion amount of a motion vector obtained by performing the motion detection, and sets the recommended reproduction speed as the reproduction control information.
 3. The image reproducing apparatus according to claim 2, wherein the reproduction control information generating unit determines the recommended reproduction speed on the basis of a result of comparison between a difference in motion amount between a region of a center portion of the captured image and a remaining region of the captured image and a preset threshold value.
 4. The image reproducing apparatus according to claim 2, further comprising an expansion processing unit that performs an expansion process on the image signal in a case where the image signal is an image signal that has been subjected to a compression process using a motion vector obtained by performing motion detection, wherein the reproduction control information generating unit generates the reproduction control information using the motion vector used in the expansion process.
 5. The image reproducing apparatus according to claim 1, further comprising a signal recording unit that records, on a recording medium, an image signal of a captured image that is generated by performing image capture at an image rate higher than the preset image rate, wherein the reproduction control information generating unit generates the reproduction control information using the image signal recorded on the signal recording unit, wherein the signal recording unit records the reproduction control information generated by the reproduction control information generating unit on the recording medium in association with the image signal, wherein the signal reproducing unit reproduces the image signal and the reproduction control information, and wherein the control unit controls an image rate of an image signal output from the signal reproducing unit on the basis of the reproduction control information reproduced by the signal reproducing unit.
 6. An image reproducing method comprising: reproducing an image signal of a captured image that is generated by performing image capture at an image rate higher than a preset image rate; generating reproduction control information using a motion detection result obtained by performing motion detection using the image signal; converting an image rate of the image signal into the preset image rate by combining frames of the image signal; controlling the image rate of the image signal on the basis of the reproduction control information to reduce a reproduction speed of the captured image in accordance with an increase in motion detected in the motion detection result; and setting weighting coefficients for respective frames of the image signal based on the motion detection result, wherein the frames are combined according to the weighting coefficients, and wherein the respective frames of the image signal are weighted with at least one of at least two weighting coefficients.
 7. An image capturing apparatus comprising: an image capturing unit that generates an image signal of a captured image that is generated by performing image capture at an image rate higher than a preset image rate; a reproduction control information generating unit that generates, using a motion detection result obtained by performing motion detection using the image signal generated by the image capturing unit, reproduction control information for reducing a reproduction speed of the captured image in accordance with an increase in motion detected in the motion detection during reproduction of the image signal, wherein the reproduction control information generating unit sets weighting coefficients for respective frames of the image signal based on the motion detection result, and the respective frames of the image signal are weighted with at least one of at least two weighting coefficients; and an output unit that outputs the reproduction control information in correspondence with the image signal.
 8. The image capturing apparatus according to claim 7, wherein the reproduction control information generating unit determines a recommended reproduction speed from a motion amount of a motion vector obtained by performing the motion detection, and sets the recommended reproduction speed as the reproduction control information.
 9. The image capturing apparatus according to claim 8, wherein the reproduction control information generating unit determines the recommended reproduction speed on the basis of a result of comparison between a difference in motion amount between a region of a center portion of the captured image and a remaining region of the captured image and a preset threshold value.
 10. The image capturing apparatus according to claim 7, further comprising a signal processing unit that converts the image signal generated by the image capturing unit into the preset image, rate, wherein the reproduction control information generating unit performs the motion detection using the image signal converted into the preset image rate by the signal processing unit.
 11. The image capturing apparatus according to claim 7, further comprising an image compression unit that compresses an amount of signal of the image signal using a motion detection result obtained by performing motion detection using the image signal, wherein the reproduction control information generating unit generates the reproduction control information using the motion detection result obtained by the image compression unit.
 12. A control method for an image capturing apparatus, comprising: generating an image signal of a captured image that is generated by performing image capture at an image rate higher than a preset image rate; generating, using a motion detection result obtained by performing motion detection using the image signal, reproduction control information for reducing a reproduction speed of the captured image in accordance with an increase in motion detected in the motion detection during reproduction of the image signal; setting weighting coefficients for respective frames of the image signal based on the motion detection result, wherein the respective frames of the image signal are weighted with at least one of at least two weighting coefficients; and outputting the reproduction control information in correspondence with the image signal. 